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REFLECTIONS  ON  BLENDED  DISTRIBUTED  LEARNING: 
THE  ARMOR  CAPTAINS  CAREER  COURSE 

INTRODUCTION 


The  importance  of  a  highly  trained  and  skilled  military  has  never  been  greater  than  today. 
Rising  to  meet  this  need  is  the  capability  to  train  personnel  anywhere  in  the  world  at  any  time 
using  distributed  learning  (TRADOC,  1999).  Cost  and  course  accessibility  are  two  key  factors 
fueling  distributed  learning  experimentation  and  development.  In  business  and  industry, 
projected  savings  of  30  to  60  percent  over  traditional  classroom  instruction  has  placed  e-leaming 
in  the  spotlight  (Fortune,  2001).  Many  questions  remain,  however,  about  the  return  on 
investment  related  to  e-leaming  expenditures  and  investments  (Raths,  2001;  Worthen,  2001).  As 
distance  learning  technology  contracts  are  announced  and  new  policies  are  enacted,  there  is  a 
growing  need  for  research  on  distributed  learning  courses  and  programs  (Bonk  &  Wisher,  2000). 

Organizations  are  devoting  increasing  time  and  energy  to  online  training  (Bonk,  2002; 
Training,  2000;  Urdan  &  Weggen,  2000).  Perhaps  the  fastest  growing  aspect  of  this  movement 
is  a  blended  approach  that  weaves  together  training  approaches  and  technologies  as  needed 
(Ganzel,  2001).  In  blended  learning,  instructors  might  embed  Web-based  instruction  with  live 
instruction,  utilize  the  Web  to  supplement  live  instruction,  or  combine  segments  of  a  Web  course 
that  are  self-paced  with  those  requiring  significant  instructor  presence  and  guidance  (Bonk, 

2002;  Rowe,  2000).  Still  another  model  would  combine  opportunities  for  live  Web-based 
instruction  with  delayed  or  asynchronous  online  instmction  as  well  as  with  face-to-face 
meetings.  Research  by  Kang  (1998),  for  instance,  revealed  that  such  combined  approaches 
impact  student  social  identity  and  relationships,  team  building,  and  decision-making,  as  well  as 
the  mentoring,  scaffolding,  and  overall  roles  of  the  instructor. 

Online  Incentives  and  Motivators 

At  the  heart  of  such  research  is  a  learner-centered  model  that  provides  choice,  meaningful 
activities,  project-based  learning,  and  opportunities  for  student  interaction  and  active  learning 
(Lotus  Institute,  1996;  Report  of  the  Commission  on  Technology  and  Adult  Learning,  2001). 
When  incorporating  a  learner-centered  model,  the  role  of  the  instructor  shifts  from  transmitter  of 
knowledge  to  that  of  facilitator  or  coach  in  the  process  (Bonk,  Wisher,  &  Lee,  in  press).  Of 
course,  instructors  have  a  myriad  of  roles  and  responsibilities  to  coordinate  for  e-leaming 
success,  including  pedagogical,  social,  organizational,  and  managerial  roles  (Anderson,  Rourke, 
Garrison,  &  Archer,  2001;  Bonk,  Kirkley,  Hara,  &  Dennen,  2001;  Mason,  1991).  A  delicate  and 
infomied  balance  between  these  roles  is  vital  to  e-leaming  success. 

With  any  new  teaching  and  learning  environment  come  numerous  challenges  and 
concerns.  One  challenge  often  mentioned  within  online  training  environments  is  motivating  and 
engaging  students  in  order  to  boost  retention  and  course  completion  rates  (Phelps,  Wells,  & 
Hahn,  1991).  While  some  claim  that  tests  show  better  student  retention  rates  for  e-leaming 
classes  than  for  traditional  instmction  (Galagan,  2001),  a  recent  survey  of  201  corporate  trainers 
and  administrators  revealed  fairly  dismal  e-leaming  completion  rates  across  many  types  and 
sizes  of  organizations  (Bonk,  2002).  Similarly,  a  study  of  asynchronous  learning  using 
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computer-mediated  communication  in  a  military  training  setting  showed  some  cost  and  learning 
improvements  over  traditional  instruction,  but  student  completion  rates  were  lower,  due,  in  part, 
to  family  and  job  commitments  (Phelps,  Wells,  et  al.,  1991).  The  extent  of  learning  gains  and 
completion  rate  differences,  however,  were  not  consistent  across  the  online  courses  (Phelps, 
Ashworth,  &  Hahn,  1991). 

In  addressing  this  predicament,  Moshinskie  (2001)  created  a  model  related  to  motivation 
in  e-leaming  environments  that  targets  the  improvement  of  learner  motivation  before,  during, 
and  after  online  courses.  He  listed  a  number  of  extrinsic  devices  (e.g.,  supportive  learning 
environments,  compensation,  certification,  paid  time  off,  gifts,  etc.)  that  might  complement 
student  intrinsic  needs.  In  terms  of  supportive  environments,  Moshinskie  noted  the  success  that 
Motorola  has  experienced  when  providing  human  contact  and  social  support  to  first-time  e- 
leamers  during  the  initial  weeks  of  an  online  course. 

In  a  recent  survey  of  201  corporate  trainers  and  training  managers,  however,  Bonk  (2002) 
found  that  most  organizations  did  not  offer  incentives  for  the  completion  of  online  courses. 
Among  those  that  did,  the  most  common  incentive  was  increased  job  responsibility.  Common 
course  activities  and  motivational  strategies  were  also  explored  in  this  study.  Motivational 
factors  perceived  as  important  in  that  study  were  the  use  of  relevant  materials,  responsive 
feedback,  goal-driven  activities,  choice  and  flexibility,  opportunities  for  personal  growth,  fun, 
interactivity  and  collaboration,  and  variety.  Specific  techniques  or  activities  that  these  trainers 
and  designers  deemed  highly  engaging  and  useful  included  case  activities  and  job  reflections, 
brainstorming,  group  tasks  and  teamwork,  electronic  mentoring  and  online  guests,  and  students 
taking  the  initiative  to  lead  discussion.  What  was  interesting  was  the  low  support  for  both  online 
conflict  and  psychological  safety  or  belongingness.  Surprisingly,  social  tasks  and  icebreakers,  as 
well  as  opportunities  to  display  or  share  products,  received  the  lowest  support  from  the  twelve 
listed  motivational  activities  in  this  particular  survey. 

Online  Benefits 

While  there  may  be  problems  related  to  online  learning  incentives  and  motivational  tools, 
many  reports  continue  to  focus  on  the  employee  and  employer  benefits  of  online  training 
technologies  and  environments.  For  instance,  Urdan  and  Weggen  (2000)  point  to  just-in-time 
access  to  information,  faster  learning  and  higher  retention  through  personalized  learning, 
substantial  cost  savings  by  eliminating  travel,  improved  interactivity  and  collaboration  among 
students,  less  intimidation  by  instructors,  and  the  ability  for  anyone  to  learn  anywhere  and  at 
anytime.  Unfortunately,  these  authors  provide  few  empirical  studies  to  back  up  these  claims. 
And,  in  certain  situations,  many  of  these  advantages  might  be  perceived  as  disadvantages  For 
example,  arguments  that  online  content  is  fresh,  consistent,  and  constantly  updated  requires 
organizations  and  instructors  to  spend  extensive  time  and  money  to  keep  their  teaching  and 
training  materials  up-to-date.  Along  these  same  lines,  the  elimination  of  travel  will  obviously 
decrease  natural  face-to-face  interaction  and  opportunities  to  share  personal  experiences  and 
relevant  stories.  Furthermore,  the  ability  to  learn  anywhere  or  anytime  expands  the  time 
requirements  of  the  course,  thereby  forcing  instructors  to  monitor  course  activities  for  extended 
hours. 

Murray  and  Bloom  (2000)  provide  a  more  research-referenced  list  of  employee  and 
employer  benefits  related  to  e-leaming.  In  terms  of  employers,  online  learning  technologies  can 
provide:  (1)  cost  savings,  (2)  flexibility  in  content  design  and  delivery,  (3)  increased  interaction 
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and  collaboration,  (4)  learning  that  is  directly  linked  to  work,  (5)  decentralized  learning,  (6) 
training  aligned  to  current  job-related  needs,  (7)  the  motivation  for  employees  to  invest  time  and 
energy  into  learning,  and  (8)  enhanced  learning  retention.  In  terms  of  employees,  online 
technologies  provide:  (1)  more  control  over  learning,  (2)  focused  and  relevant  learning  matched 
to  individual  learning  needs,  (3)  skills  that  increase  one’s  value  to  the  organization,  (4)  improved 
self-confidence,  (5)  new  competencies  that  enhance  job  satisfaction,  (6)  skills  that  boost  job 
productivity  and  performance,  (7)  mechanisms  for  tracking  and  recognizing  achievement,  (8) 
information  leading  to  safer  work  environments,  and  (9)  opportunities  to  use  learning 
technologies  that  bolster  one’s  e-literacy.  Naturally,  Murray  and  Bloom  discuss  many  challenges 
within  these  environments,  including  technology  limitations,  system  difficulty,  measurement 
failures,  management  resistance  to  change,  poor  planning  and  direction,  a  lack  of  innovation 
champions,  learner  resistance  to  online  training,  and  a  lack  of  time,  money,  and  support. 

The  purpose  of  this  research  is  to  understand  how  a  blended  or  hybrid  approach  to  e- 
leaming  impacted  the  professional  development  of  students  in  a  high-level  military  training 
course.  The  research  addresses  e-leaming  from  the  perspectives  of  the  course  learners,  the  DL 
Education  Advisor,  and  the  instructors.  Interviews  with  these  individuals  helped  document 
distinct  advantages  and  disadvantages  from  different  components  of  the  course.  Issues  and 
considerations  for  e-leaming  mentioned  consistently  across  groups  should  help  with  future 
course  design  and  delivery  methods.  In  effect,  this  research  might  help  in  forming  instructional 
design  principles  for  the  Web  as  well  as  in  the  fine-tuning  of  this  particular  program  and  ones 
similar  to  it. 
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RESEARCH  OVERVIEW 


Background  Information 

At  the  U.S.  Army  Armor  School  in  Fort  Knox,  Kentucky,  the  use  of  collaborative 
learning  environments  is  taking  center  stage  in  all  phases  of  the  Armor  Captains  Career  Course 
(AC3-DL)  (Wardell  &  Paschetto,  2000).  In  part,  this  form  of  training  is  meant  to  be  a  low-cost 
alternative  to  other  common  training  practices.  In  addition,  it  is  intended  to  offer  more 
flexibility,  choice,  interactivity,  and  tracking  than  the  previous  use  of  a  correspondence  course 
combined  with  a  final  two-week  residential  training  program.  The  purpose  of  the  AC3-DL  is  to 
train  captains  to  command  companies  and  perform  as  assistant  operations  officers  at  command 
units  such  as  a  battalion.  The  course  covers  advanced  leadership  skills  from  planning  combat 
missions  to  handling  the  supplies,  maintenance,  and  information  assets  of  a  complex 
organization.  The  targeted  population  is  first  lieutenants  or  captains  with  four  to  six  years  of 
military  service,  often  in  their  late  20s  or  early  30s.  In  effect,  AC3-DL  provides  the  necessary 
knowledge  and  skills  for  mid-level  management  of  future  military  operations.  The  AC3-DL 
training  is  conducted  in  three  phases;  the  first  two  are  online  (Phase  la:  asynchronous,  and  then 
Phase  lb:  synchronous),  while  the  final  phase  is  face-to-face. 

Phase  la:  Asynchronous  Learning 

The  first  phase  of  AC3-DL  is  the  asynchronous  component  during  which  students  learn 
basic  terms  and  concepts  via  the  Internet  with  both  computer  and  instructor  feedback.  This  self- 
paced  stage,  designed  to  be  equivalent  to  a  three-credit,  college-level  course,  contains  animation, 
interactive  audio  and  video,  and  historical  tracking  of  learner  progress  through  each  module. 

The  content  was  estimated  to  be  equal  to  about  240  hours  of  instruction  with  an  instructor 
moderating  and  providing  feedback  on  student  progress.  Delivery  of  this  instruction  was 
intended  to  take  about  one  year,  although  highly  motivated  students  can  complete  it  in  less  time. 

Each  lesson  has  a  set  of  objectives  consisting  of  actions,  conditions,  and  standards.  To 
determine  how  well  students  are  meeting  these  objectives,  they  are  tested  before  and  after  each 
lesson  as  well  as  at  the  end  of  a  complete  volume  of  lessons.  The  lesson  tests  are  multiple  choice 
and  graded  by  the  computer.  End  of  volume  questions  are  embedded  in  longer  “gate”  tests  that 
include  both  computer-scored  multiple  choice  tests  and  instructor-graded  problem-solving 
scenarios  revolving  around  mission  statements,  decision  matrices,  and  alternative  courses  of 
action.  A  student  must  earn  70  percent  or  higher  on  each  part  of  a  gate  test  before  the  instructor 
will  open  the  gate  and  pass  him  through  to  the  next  volume  or  module.  Students  can  retake  end 
of  volume  tests  until  they  obtain  scores  of  70  percent  or  higher.  With  the  historical  tracking 
within  the  learning  management  system  (see  Figure  1),  instructors  can  determine  the  modules 
and  components  within  particular  modules  where  students  are  experiencing  the  most  difficulty, 
as  well  as  the  present  status  of  students  in  the  course.  In  effect,  instructors  can  project  student 
attrition  and  completion  rates,  thereby  allowing  allow  them  to  begin  planning  for  Phases  Ih  and 
II. 
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Figure  1.  Example  of  student  history  files  in  the  asynchronous  component  of  the  course. 
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Phase  lb:  Synchronous  Learning 

In  the  second  phase,  or  synchronous  component,  students  are  placed  into  small  groups  or 
teams  with  other  individuals  from  across  the  country  while  working  in  the  Virtual  Tactical 
Operations  Center  (VTOC),  which  contains  a  set  of  software  tools  developed  specifically  for  this 
course.  VTOC  contains  seven  extensions  (or  rooms).  One  of  these  seven  rooms  is  the  “Main” 
extension,  which  opens  to  six  others  —  three  on  each  side.  Every  extension  or  room  has  the  same 
collaborative  tools,  making  it  possible  for  smaller  groups  to  collaborate  independently  of  the 
main  group.  Only  one  tool  can  be  in  use  in  each  extension  at  any  time.  Therefore,  if  two  students 
wanted  to  share  the  bookshelf,  and  another  wanted  to  collaborate  on  an  overlay,  they  would  have 
to  move  to  different  extensions. 

During  this  phase,  the  students  collaborated  in  the  VTOC.  As  indicated  below, 
collaboration  occurs  through  a  live  audio  connection,  real-time  online  chats,  and  the  use  of 
various  software  tools.  There  are  four  general  ways  to  collaborate: 

1 .  Every  student  appears  as  an  avatar  or  virtual  image  in  the  3D  world,  so  others  can  see  his 
"location." 

2.  Every  student  participates  in  an  audio  conference  with  others  in  the  same  extension  (or 
room)  of  the  VTOC.  This  audio  conference  allows  anyone  to  speak  at  any  time,  and 
everyone  else  will  hear  him.  The  maximum  number  of  participants  is  15. 
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3.  Every  student  has  several  text  chat  windows  -  one  for  the  particular  extension  they  are  in, 
one  that  is  global,  and  another  private  chat  they  can  open  by  invitation  that  is  shared  with 
only  one  other  person. 

4.  Every  student  can  access  various  specially  designed  tools  for  collaboration  (shared 
applications). 

There  are  several  unique  collaborative  tools  available  in  the  VTOC:  (1)  shared  text,  (2) 
shared  bookshelf,  (3)  Mapedit,  and  (4)  3D  terrain.  These  collaborative  tools  have  some 
characteristics  in  common.  For  example,  the  first  user  to  open  the  tool  becomes  that  tool's  driver 
(until  the  user  relinquishes  control  by  closing  the  tool).  The  driver  chooses  which  file  is  to  be 
opened,  and  if  the  work  is  saved,  the  driver  names  the  file.  In  the  case  of  shared  text,  the  driver  is 
the  only  one  who  enters  text  —  others  (the  followers)  can  only  read  what  is  written. 

The  shared  text  application  consists  of  shared  HTML  forms1 .  Such  forms  for  shared  text 
application  help  students  to  write  operations  orders,  warning  orders,  and  other  products  that  are 
part  of  the  planning  process.  The  shared  bookshelf2,  in  contrast,  is  used  for  displaying  field 
manuals  or  “slide  shows"  that  someone  may  want  to  review.  The  third  tool,  the  Mapedit 
program3,  was  developed  to  create  map  overlays,  emulating  plastic  sheets  on  which  symbols  are 
drawn  that  are  laid  onto  a  map  (see  Figure  2).  If  students  want  a  whiteboard,  they  simply  open  a 
blank  overlay  (no  map  background).  Fourth,  the  3D  terrain  is  a  collaborative  environment  that 
does  not  result  in  a  product,  but,  instead,  enables  students  and  instructors  to  "walk"  the  terrain 
and  lay  an  overlay  on  the  ground.  Such  a  tool  provides  users  a  different  way  to  visualize  their 
plans.  Participants  can  click  on  another  person's  avatar,  and  they  will  see  what  that  person  sees. 

In  this  way,  an  instructor  can  take  a  group  of  students  on  a  walk  through  the  terrain,  certain  that 
their  view  will  be  the  same  as  his. 


1  The  “driver”  of  the  shared  text  application  enters  text,  and  chooses  what  "page"  of  the  text  form  is  visible. 
Followers  are  read  only.  However,  followers  can  contribute  to  the  content  via  either  text  chat  or  voice  conference. 

2  In  the  shared  bookshelf,  the  driver  chooses  what  HTML  document  is  visible,  and  followers  then  see  that  document. 

3  Mapedit  allows  multiple  users  to  add,  delete,  and  move  symbols  and  lines  on  the  map  overlay.  In  Mapedit,  the 
driver  chooses  which  file  to  open,  and  names  the  file  to  save,  but  all  users  can  edit  the  contents.  Mapedit  is  used  to 
share  both  overlays  and  other  sketches  (the  only  difference  between  Mapedit  and  a  shared  whiteboard  is  that  an 
overlay  has  a  map  behind  it). 
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Figure  2.  Example  of  Mapedit  feature  in  the  VTOC. 
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Using  these  collaborative  tools,  students  worked  with  each  other  and  the  instructor  for  10 
weekends  (or  roughly  60  hours  of  asynchronous  and  120  hours  of  synchronous  instruction).  In 
asynchronous  mode,  they  acquired  important  background  knowledge,  while  synchronously  they 
engaged  in  a  host  of  collaborative  training  exercises  that  resulted  in  a  finished,  doctrinally  sound 
product.  Synchronous  collaborations  were  scheduled  in  advance  and  held  during  weekends. 
While  asynchronous  collaborations  can  occur  through  the  use  of  e-mail,  synchronous 
collaboration,  as  noted  earlier,  includes  the  use  of  a  shared  whiteboard  for  map  editing,  text  chat 
with  voice  over  IP  (2  way  and  multipoint),  the  shared  bookshelf,  private  chat,  and  the  3D  terrain 
tool.  Students  also  have  access  to  a  shared  version  of  Microsoft  Word,  which  includes 
opportunities  for  group  editing  and  other  collaborative  writing  activities. 

The  synchronous  phase  centers,  in  part,  on  groups  of  10-12  students  acting  as  battalion 
staff  officers  working  together  to  build  an  operations  order  based  on  different  staff  positions  that 
they  have  been  assigned  to.  In  these  role-play  situations,  they  engage  in  activities  to  create, 
share,  and  evaluate  tactical  plans.  Such  plans  might  address  actions  (e.g.,  critical  events  and 
decision  points),  direct  and  indirect  reactions,  intelligence,  maneuvers,  fire  support,  mobility, 
logistics,  command  and  control,  and  other  related  items.  One  activity  often  used  is  a  mission 
analysis  which  includes  information  gathering  and  critical  reflection  on  terrain  and  weather, 
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enemy  forces,  facts,  assumptions,  limitations,  specific  tasks,  implied  tasks,  assets  available, 
additional  considerations,  and  a  proposed  restated  mission  (see  Figure  3).  During  this  real-time 
training,  AC3-DL  instructors  help  students  work  through  their  tactical  maneuvers  and  other 
decision-making  activities.  Given  the  complexity  and  deemed  importance  of  this  phase,  studies 
are  needed  to  examine  the  collaborative  and  interactive  nature  of  these  training  events. 

Figure  3.  Example  of  a  mission  analysis  in  the  VTOC 
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Phase  II:  Residential  Learning 

The  third,  and  final,  phase  of  AC3-DL  took  place  at  Fort  Knox  where  the  groups  met 
face-to-face  for  the  first  time.  Here  they  engaged  in  traditional  classroom  and  field  exercises. 
This  two-week  phase  of  group-paced  instruction  was  delivered  in  a  classroom,  through 
constructive  and  virtual  simulations,  and  on  terrain.  In  effect,  it  was  a  capstone  experience 
where  students  were  challenged  to  apply  the  skills  and  competencies  that  they  had  acquired  via 
distance  learning  (Wardell  &  Paschetto,  2000). 

These  three  phases  were  designed  to  move  from  a  focus  on  individual  knowledge  to  small 
group  collaboration  and  application  skills  to  collective  unit  problem  solving  and  decision¬ 
making. 
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Previous  Research 


Prior  to  this  research,  a  few  interesting  research  projects  were  conducted  on  AC3-DL. 

For  instance,  Sanders  and  Guyer  (2001)  conducted  a  14-item  survey  of  students  and  unit  leaders 
involved  in  the  AC3-DL  program.  The  open-ended  items  in  this  survey  related  to  factors 
negatively  affecting  student  participation  in  this  program.  In  particular,  this  survey  inquired 
about  policies,  monetary  incentives,  course  impact  on  other  student  responsibilities,  and  factors 
that  contributed  to  attrition.  While  the  response  rate  was  relatively  low  (33  of  208  student 
surveys  requests  and  31  of  96  unit  leader  surveys  requests  were  completed),  there  were  some 
interesting  trends  that  relate  to  findings  reported  in  the  present  research.  For  instance,  both 
students  and  unit-leaders  were  fairly  positive  about  the  AC3-DL  program  and  noted  that  it  was 
an  improvement  over  the  previous  paper-based  correspondence  modes  of  delivery.  Student 
attitudes  about  the  benefits  of  Web-based  instruction  were  generally  more  favorable  than  those 
from  unit  leaders. 

The  survey  also  revealed  problems  with  the  length  of  some  of  the  course  modules.  In 
addition,  several  students  reported  time-related  conflicts  between  their  standard  unit  drill  times 
and  AC3-DL  course  time  requirements.  While  one’s  present  employment  situation,  technical 
problems  with  equipment,  family  responsibilities,  and  the  course  format  were  common 
asynchronous  factors  limiting  participation,  the  key  factors  hindering  synchronous  course 
participation  included  employment,  completing  unit  drill  requirements,  technology  problems 
with  equipment,  course  time  requirements,  and  lack  of  compensation.  Importantly,  most  of  the 
students  reported  being  very  familiar  with  asynchronous  tools  for  communication  and  at  least 
somewhat  familiar  with  synchronous  chat  tools.  Various  recommendations  were  made  in  this 
report  concerning  technology  training,  technical  support,  compensation,  supplemental  media, 
and  lesson  length. 

A  second  study  by  Sanders  and  Burnside  (2001)  was  conducted  on  the  asynchronous 
component  of  the  course  to  determine  whether  students  in  the  AC3-DL  program  learned  as 
effectively  as  those  in  the  pen  and  paper  correspondence  course.  These  researchers  found  that 
students  in  the  Web  version  of  the  course  completed  their  training  in  less  time  than  those  in  the 
correspondence  version.  Additionally,  student  and  instructor  surveys  and  interviews  regarding 
the  Web  version  of  the  course  were  generally  positive  compared  to  paper-based  correspondence 
courses.  In  fact,  the  study  revealed  that  there  was  content  covered  in  the  Web  version  of  the 
course  that  was  not  addressed  in  the  correspondence  version.  Small  group  instructors  indicated 
that  students  trained  via  the  Web  were  more  likely  to  make  effective  decisions  and  develop  a 
greater  sense  of  team  identity  than  the  correspondence  students.  In  addition,  Web  students 
demonstrated  more  organized  planning,  confidence  in  front  of  their  peers,  tactical  proficiency, 
and  general  leadership  and  supervisory  skills.  A  comparison  of  student  knowledge  in  company 
team  operations  favored  the  Web  group  but  was  not  significant.  Nevertheless,  there  were  student 
complaints  concerning  the  length  of  some  of  the  AC3-DL  modules  and  training  components  as 
well  as  several  problems  and  limitations  with  some  of  the  technologies  utilized  in  the  program. 
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METHODS  AND  PROCEDURE 


Background  Information 

Months  prior  to  the  focus  group  sessions  conducted  for  this  research,  the  DL  Education 
Advisor  and  key  assistants  shared  background  information  about  the  program  and  the  technology 
during  two  separate  meetings  with  the  researchers.  At  those  times,  these  individuals 
demonstrated  integral  aspects  of  the  distance  learning  tools  as  well  as  samples  of  the  text 
transcripts  and  associated  instructor  feedback.  It  was  clear  from  these  demonstrations,  reports, 
and  handouts  that  there  was  immense  course  planning  and  monitoring  related  to  this  course.  All 
personnel  involved  in  this  project  seemed  genuinely  interested  in  determining  the  impact  and 
effectiveness  of  this  new  delivery  mechanism.  They  were  clearly  focused  on  fostering  student 
success.  Hence,  as  a  design  team,  they  focused  on  creating  a  student-centered  training 
environment. 

Their  reports  noted  that,  from  an  historical  standpoint,  distance  learning  within  the  Army 
had  undergone  a  series  of  transformations.  Such  training  had  included  the  use  of  paper-based 
correspondence  during  the  1940s  to  the  early  1990s,  televised  teams  during  the  1950s  to  the 
1960s,  film  strips  in  the  1970s,  videodiscs  in  the  1980s,  video-teletraining  from  1992  to  1996, 
CD-ROM-based  instruction  beginning  in  the  early  1990s,  and  Internet-based  courses  starting  in 
1997.  In  effect,  AC3-DL  was  promoted  as  cutting  edge  courseware  designed  and  developed  for 
the  Internet.  Importantly,  the  new  Web-based  course  format,  which  received  national 
recognition  for  excellence  in  distance  learning  (Warded  &  Paschetto,  2000),  was  based  on  recent 
cognitive  and  instructional  design  principles. 

There  were  many  unique  aspects  related  to  this  courseware.  For  instance,  it  was  intended 
to  take  advantage  of  the  strengths  of  different  delivery  mechanisms  including  chat,  e-mail, 
synchronous  chats,  virtual  worlds,  simulations,  etc.  One  key  component,  the  learner 
management  system,  was  designed  to  provide  useful  and  timely  historical  data  and  visual 
depictions  of  the  modules  that  students  completed.  With  this  system,  the  progress  of  students 
through  the  asynchronous  phase  of  the  program,  including  testing  activities,  could  be  tracked  and 
monitored.  Instructor  monitoring  and  evaluation  of  student  progress  combined  with  timely 
feedback  and  the  ability  to  earn  college  credit  was  intended  to  reduce  student  attrition  in  the 
course.  For  instance,  instructors  were  asked  to  provide  e-mail  feedback  within  24  hours  and  gate 
testing  feedback  within  72  hours.  Instructor  involvement  and  individualized  attention  were  key 
aspects  of  the  course  design.  Such  involvement  naturally  requires  additional  instructor  training 
and  support  (Sanders  &  Burnside,  2001). 

To  foster  a  learner-centered  environment,  the  content  and  activities  were  selected  based 
on  real-world  situations  and  authenticity.  Furthermore,  different  forms  of  media  (e.g.,  audio, 
video,  and  animation)  were  intended  to  address  a  variety  of  student  learning  styles.  As  alluded  to 
earlier,  both  asynchronous  and  synchronous  training  components  were  selected  to  address 
different  learning  needs.  For  instance,  research  indicates  that  while  synchronous  instruction 
might  facilitate  two-way  interaction  and  socio-emotional  interaction,  asynchronous  instruction 
tends  to  be  used  for  one-way  task  completion  efforts  (Chou,  2000).  Additionally,  as  shown  by 
Bonk,  Hansen,  Grabner-Hagan,  Brown,  and  Mirabelli  (1998),  while  real  time  chat  tools  foster 
more  responses  per  student  and  increased  peer  interaction,  asynchronous  tools  promote  idea 
development  and  evaluation  in  greater  depth. 
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In  this  research,  asynchronous  tools  included  audio,  video,  and  animation  components 
meant  to  train  complex  cognitive  skills.  To  enhance  and  perhaps  complement  the  emergence  of 
new  cognitive  skills  and  attitudes,  synchronous  tools  allowed  students  to  create,  display,  and 
share  digital  overlays  and  maps  as  well  as  communicate  with  peers  and  instructors.  Early  phases 
of  the  instruction  were  designed  to  foster  particular  student  abilities  (e.g.,  classification,  writing, 
preparation,  decision-making,  planning,  and  critique  skills),  whereas  later  phases  targeted  other 
competencies  (e.g.,  analysis,  evaluation,  monitoring,  and  specification  skills).  In  assessments  of 
those  abilities,  students  received  immediate  feedback  from  a  series  of  automated  pre-tests,  post¬ 
tests,  and  practical  exercises.  Learner  engagement  was  addressed,  in  part,  through  more  random 
tools  (such  as  those  they  labeled  “firefights”).  In  an  attempt  to  further  motivate  students,  many 
activities  also  had  elements  of  fun,  humor,  and  dissonance  embedded  in  them.  As  motivational 
theorists  point  out  (e.g.,  Reeve,  1996),  such  elements  are  often  intrinsically  motivating  for 
learners.  Clearly,  in  all  phases,  the  focus  was  on  assisting  the  learner  to  complete  the  AC3-DL 
course. 

Interviews 

In  the  present  research,  the  purpose  of  the  interviews  was  to  gain  a  better  understanding 
of  the  distance  learning  experience  from  both  the  instructors’  and  students’  perspectives.  The 
DL  Education  Advisor  also  provided  valuable  information  including  her  views  on  the  overall 
goals  of  AC3-DL.  A  primary  interest,  of  course,  was  simply  to  assess  the  students’  general 
reactions  to  the  course.  While  such  evaluation  is  aimed  at  the  first  level  of  Kirkpatrick’s  (1998) 
evaluation  framework  (i.e.,  student  reactions),  instructor  observations  were  valuable  in  detailing 
how  the  distance-learning  program  impacted  student  conceptual  learning  (i.e.,  Level  2  of  the 
Kirkpatrick  model)  as  well  as  the  transferability  of  skills  learned  online  to  real-world  settings 
(i.e.,  Level  3  of  the  Kirkpatrick  model). 

In  June  2001,  interviews  were  conducted  with  eight  students  and  three  instructors 
recently  involved  the  AC3-DL.  In  addition,  the  DL  Education  Advisor  for  the  Armor  School, 
who  helped  design  the  program,  was  interviewed.  Most  of  these  interview  sessions  were 
approximately  one  hour  in  length.  The  eight  students,  who  were  members  of  the  Army  National 
Guard,  also  completed  a  series  of  short  questionnaires  related  to  their  backgrounds  and 
experience  with  computer  technology,  the  online  learning  environment,  and  the  overall 
effectiveness  of  Phase  lb  training  (including  workgroup  attitudes,  satisfaction,  efficacy,  and 
interpersonal  as  well  as  task  cohesion). 
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RESULTS 


Questionnaire  Responses 

Though  somewhat  small  in  number,  the  eight  students  that  participated  in  the  focus 
groups  had  vital  perspectives  and  experiences  since  they  had  devoted  several  hundred  hours  to 
the  distance-learning  format  of  instruction.  Important  to  an  online  course,  all  these  individuals 
indicated  that  they  had  a  personal  computer  available  for  their  use  and  had  access  to  the  Internet 
at  both  their  home  and  work  settings.  Of  the  eight  students,  two  participants  (25%)  believed  that 
in  general,  Internet  instruction  was  more  effective  than  classroom  instruction,  while  six 
participants  (75%)  believed  that  classroom  instruction  was  more  effective.  Overall,  the  students 
favorably  evaluated  their  experience  working  in  groups  online  and  were  extremely  satisfied  with 
the  synchronous  portion  of  the  course.  The  students’  responses  also  indicated  high  levels  of  both 
individual  efficacy  and  collective  efficacy  with  the  synchronous  portion  of  the  course,  and  high 
degrees  of  both  interpersonal  and  task  cohesiveness  within  their  teams.  In  addition,  twenty  items 
were  adopted  from  a  scale  to  measure  the  degree  to  which  the  online  environment  reflected  a 
social  constructivist  learning  community  (Bonk  &  Wisher,  2000).  Interestingly,  participants 
tended  to  indicate  that  the  environment  represented  an  active  or  social  constructivist  community 
(average  rating  of  5.0  on  a  scale  of  1  (Strongly  Disagree)  to  7  (Strongly  Agree)). 


Interviews  with  Student  Focus  Groups 


Technological  Obstacles 

During  the  focus  group  sessions,  it  was  apparent  that  there  were  several  technological 
problems  and  obstacles  related  to  the  AC3-DL  course.  The  most  common  technological 
problems  that  the  students  faced  involved  the  virtual  chat  rooms  and  the  map  editor.  Such 
technological  problems  can  frustrate  students,  interfere  with  learning,  and,  ultimately,  negatively 
impact  student  course  completion  rates. 

Course  Attrition  and  Incentives 

Attrition  is  a  pervasive  problem  in  distance-learning  courses  in  higher  education  settings 
(Bonk,  2001 ;  Phipps  &  Merisotis,  1999)  as  well  as  in  training  environments  (Bonk,  2002). 
Consequently,  this  was  an  issue  addressed  in  the  focus  group  discussions.  During  the  two  focus 
group  sessions,  the  students  were  asked  if  they  had  ever  considered  dropping  out  and  why. 

Seven  of  the  eight  students  had  never  considered  dropping  out  because  they  needed  the  course 
for  career  progression.  Their  answers  reflected  a  key  difference  between  military  courses  and 
many  non-military  courses— military  students  must  complete  specific  courses  if  they  want  to 
advance  in  their  careers.  As  an  indicator  of  the  importance  of  course  completion  incentives,  the 
eight  AC3-DL  students  interviewed  here  were  sufficiently  motivated  due  to  opportunities  for 
career  progression. 
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Learning  Environment 

The  results  from  the  focus  group  discussions  revealed  that  the  overall  learning 
environment  helped  students  throughout  the  course.  One  area  where  the  two  focus  groups 
differed  involved  student  perceptions  of  the  group  environment  within  the  AC3-DL  program. 
One  focus  group  felt  that  although  they  had  never  met  face-to-face  during  the  synchronous 
portion  of  the  course,  they  were  truly  a  team.  These  students  stated  that  they  engaged  in  small 
talk  and  shared  personal  information  while  they  chatted.  Such  pedagogical  activities  enabled 
them  to  begin  to  understand  the  other  group  members’  personalities,  strengths,  and  weaknesses. 
In  addition,  all  students  felt  that  allowing  different  students  to  assume  various  leadership  roles 
helped  them  learn  to  be  “followers”  and  trust  others  as  well  as  recognize  the  unique  talents  and 
strengths  of  others  in  certain  knowledge  areas.  One  student  stated  that  the  text  chat  enhanced  the 
development  of  his  relationship  with  the  group  because  he  had  to  reflect  deeply  on  what  he  was 
going  to  “say”  before  typing  and  could  do  so  without  interruptions.  However,  in  the  second 
focus  group,  the  students  stated  that  they  did  not  feel  as  much  of  a  sense  of  camaraderie,  but, 
instead,  felt  like  “individuals  struggling  to  work  together  as  a  team.”  They  indicated  that,  while 
developing  collaborative  products  enhanced  group  cohesiveness  by  creating  a  common  goal, 
they  often  felt  somewhat  detached  from  the  other  participants  and  would  not  “stick  their  necks 
out”  for  other  group  members. 

Instructor  Role 

Even  though  there  were  differences  in  the  perceptions  of  the  overall  learning 
environment,  the  eight  students  overwhelmingly  viewed  the  role  of  the  instructor  as  a  facilitator 
rather  than  a  lecturer  (see  Coomey  &  Stephenson,  2001  for  differences).  In  accordance  with  the 
original  program’s  design  goals,  a  key  role  of  the  online  instructors  was  to  provide  direction  and 
guidance  that  facilitated  learning.  Thus,  feedback  from  the  instructor  was  considered  important. 
Not  surprisingly,  the  students  noted  that  they  sought  feedback  on  a  consistent  basis  throughout 
the  course  in  order  to  assess  their  progress.  Another  popular  pedagogical  technique  was  the  use 
of  early  online  introductions.  Finally,  while  instructors  coordinated  the  chats,  they  made  a 
concerted  effort  to  get  everyone  to  contribute  to  the  real-time  discussions. 

Perceived  Advantages 

Given  the  fact  that  most  of  these  individuals  had  full-time  jobs  and  families,  it  was  not 
surprising  that  the  primary  advantages  mentioned  for  taking  the  course  online  versus  in  a 
traditional  classroom  were  flexibility  and  convenience.  Along  these  same  lines,  the  ability  to 
work  at  their  own  pace  and  to  be  able  to  ask  questions  without  feeling  embarrassed  or  self- 
conscious  were  vital.  Equally  important  was  the  “active”  learning  environment  that  was 
embedded  in  the  course.  Students  appreciated  the  immediate  feedback  and  mentoring  from 
instructors  about  their  progress.  Most  participants  were  not  hesitant  to  contact  the  instructor  via 
e-mail  for  such  feedback.  Whereas  in  the  asynchronous  portion  of  the  course  there  was  minimal 
interaction,  the  synchronous  portion  provided  consistent  feedback  and  interaction.  Some 
students  also  appreciated  the  fact  that,  under  special  arrangements,  the  course  could  count 
towards  credit  for  an  advanced  university  degree.  Others  mentioned  that  they  learned  to  work  as 
a  team  while  online.  Still  others  felt  that  the  online  environment  fostered  more  thoughtful 
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comments  than  normally  found  in  conventional  classroom  settings.  They  argued  that  online 
environments  encouraged  more  active  learning  than  they  would  have  experienced  in  a 
correspondence  course. 

Perceived  Disadvantages 

The  participants  stated  that  a  key  disadvantage  of  the  course  involved  the  length  of  the 
asynchronous  and  synchronous  course  components.  As  both  the  Sanders  and  Guyer  (2001)  and 
Sanders  and  Burnside  (2001)  reports  revealed,  the  students  felt  that  the  asynchronous  portion  of 
AC3-DL  was  too  long  and  filled  with  too  much  detailed  information.  This  situation  led  most  of 
the  students  to  skip  portions  of  Phase  la  just  to  get  through  it  in  a  timely  manner,  whereas  many 
of  their  classmates  simply  dropped  the  course4.  One  group  added  that  the  tests  for  each 
completed  section  were  confusing  since  they  utilized  different  formats.  In  addition,  when 
students  did  complete  a  volume  or  module,  they  could  not  proceed  without  instructor  permission, 
thereby  taking  away,  at  least  in  part,  from  the  “learn  anywhere,  learn  anytime”  benefit  of  online 
learning. 

In  contrast,  they  claimed  that  the  synchronous  portion  was  too  short.  Some  students,  for 
instance,  mentioned  that  this  phase  did  not  enable  every  group  member  to  play  a  different  role  in 
the  simulated  exercises.  They  suggested  that  additional  VTOC  sessions  would  allow  all 
members  to  experience  the  XO  (i.e.,  Executive  Officer)  and  S-3  (Operations  and  Training) 
positions.  And  while  they  claimed  to  definitely  benefit  from  this  phase  of  their  training,  the 
VTOC  tool  would,  at  times,  mysteriously  remove  students  from  the  chat  rooms,  thereby  losing 
valuable  training  time  while  disrupting  workflow.  In  addition,  the  map  editing  tool  tended  to 
freeze  some  computer  systems,  while  the  terrain  viewer  was  not  utilized  as  much  as  perhaps 
anticipated.  When  it  did  work,  most  students  did  not  like  the  use  of  the  avatar,  which 
represented  their  physical  presence  in  the  operations  areas  within  the  VTOC.  Finally,  the  VTOC 
was  not  compatible  with  the  Macintosh  computer  platform.  To  alleviate  some  of  these  problems, 
these  eight  students  recommended  that  the  course  designers  and  instructors  focus  on  basic 
technology  shown  to  function  effectively  rather  than  the  latest  gadgets  or  innovations.5  As  Scott 
(2001)  points  out,  using  a  new  technology  (e.g.,  voice  over  the  Internet)  can  be  a  mistake  if  it 
does  not  add  perceived  relevance  and  effectiveness  to  the  course  or  solve  a  key  problem.  At  the 
same  time,  students  reported  positive  experiences  with  the  voice  chat  and  e-mail  communication. 

In  addition  to  technological  concerns,  there  were  a  few  other  disadvantages  mentioned. 
For  instance,  the  online  examinations  offered  minimal  feedback  other  than  test  scores.  Students 
also  stated  that  some  of  the  modules  could  have  been  delivered  and  tested  in  smaller  chunks, 
thereby  focusing  on  specific  accomplishments.  It  is  conceivable  that  such  an  approach  would 
have  resulted  in  lower  attrition  rates  as  students  would  not  be  overloaded  with  information,  but 
would  be  able  to  achieve  a  sense  of  accomplishment  early  in  the  course.  A  couple  of  students 
mentioned  that  there  were  uneven  policies  about  the  time  they  spent  in  synchronous  weekend 


4  While  the  exact  percentages  were  not  stated,  initial  attrition  rates  for  Phase  la  of  AC3-DL  was  50  percent  for 
students  who  actively  participated,  and  approached  75  percent  if  all  students  were  included  (Sanders  &  Burnside, 
2001).  Discussions  with  AC3-DL  course  developers  and  instructors  indicated  that  the  attrition  rate  was  significantly 
reduced  in  later  cohorts  as  the  course  delivery  methods  were  modified. 

5  This  is  not  surprising  since  research  shows  that  online  instructors  and  students  tend  to  rely  on  simple  tools  such  as 
e-mail,  static  or  dynamic  syllabi,  Web  links  to  course  material,  posting  lecture  notes  online,  and  accepting  student 
work  online,  while  significantly  fewer  use  online  chatrooms,  multimedia  lectures,  online  examinations,  animation, 
and  video  streaming  (Bonk,  2001;  Peffers  &  Bloom,  1999) 
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training,  with  some  students  receiving  additional  compensation  or  credit  for  this  time.  These 
participants  argued  that  they  should  be  compensated  for  attending  the  synchronous  training  in 
lieu  of  completing  unit  drill  time  requirements.  Given  that  previous  survey  research  on  this 
course  from  Sanders  and  Guyer  (2001)  reported  similar  course  incentive  and  module  length 
issues,  this  is  an  area  that  warrants  further  attention. 

Overall  Impressions  and  Suggestions 

Overall,  the  students  enjoyed  the  distance-learning  course  and  deemed  the  technology  as 
truly  “excellent.”  In  fact,  the  only  person  who  contemplated  dropping  out  expressed  time-related 
concerns.  The  students  genuinely  appreciated  the  course  flexibility  as  well  as  the  ability  to  work 
at  their  own  pace  during  Phase  la.  Individuals  in  one  group  claimed  that  the  skills  learned  during 
the  synchronous  training  of  Phase  lb  readily  transferred  to  the  Phase  II  residential  live 
instruction.  However,  some  individuals  in  the  other  group  claimed  that  their  online  learning 
activities  did  not  transfer  since  Phase  lb  criteria  were  at  a  lower  standard  than  those  experienced 
during  Phase  II.  They  argued  that  their  document  development  activities  were  not  realistic.  Not 
surprisingly,  these  individuals  felt  that  the  most  learning  occurred  in  the  residential  portion  of  the 
program.  They  also  thought  that  they  worked  the  hardest  in  the  residential  phase  since  the 
synchronous  sessions  had  definitive  starting  and  ending  points. 

While  most  of  these  particular  learners  did  not  consider  dropping  out,  they  still  felt  that 
the  course  needed  to  be  slightly  restructured  to  further  facilitate  learning.  Recommendations 
related  to  such  structuring  included  more  lectures  and  direct  instruction  before  the  synchronous 
portion  of  the  course  to  provide  a  stronger  knowledge  foundation  from  which  to  draw  upon.  At 
the  same  time,  participants  proposed  shortening  the  asynchronous  phase  of  instruction  by  having 
fewer  practical  exercises,  decreasing  items  on  gate  tests,  and  placing  more  attention  on  the 
quality  of  knowledge  application  rather  than  quantity.  Another  suggestion  was  a  pre-orientation 
session  to  acquaint  students  with  course  expectations  and  tools,  while  simultaneously  answering 
their  questions  and  concerns.  Along  these  same  lines,  students  suggested  that  proponents  of  the 
course  find  a  way  to  display  the  course  to  battalions  around  the  United  States,  thereby  prompting 
their  interest  in  the  course  as  well  as  an  awareness  of  course  requirements.  Finally,  while  the 
majority  of  the  students  argued  that  this  course  was  best  presented  online,  all  students  felt  that 
the  face-to-face  portion  of  the  course  was  still  vital  because  that  was  where  it  “all  came 
together.” 


Interviews  with  Course  Instructors 


Instructional  Role  and  Philosophy 

Interviews  with  the  course  instructors  provided  further  insight  into  the  strengths  and 
weaknesses  of  teaching  this  complex,  Web-based  course.  First,  the  online  instructors  did  not 
view  their  roles  as  much  different  than  that  of  regular  classroom  instructors.  While  they  felt  that 
they  served  more  of  a  facilitative  role,  providing  students  with  the  means,  tools,  and  guidance  to 
learn  effectively,  they  contended  that  the  only  major  differences  between  teaching  online  and 
teaching  in  a  classroom  were  that  they  could  not  see  their  students  and  instead  of  writing  grades 
on  students’  assignments,  they  sent  them  e-mails.  Although  two  instructors  emphasized  that  they 
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allocated  the  bulk  of  their  time  to  course  planning,  the  third  instructor  indicated  that  he  had  spent 
much  more  time  on  the  administrative  aspects  of  the  course.  Nonetheless,  they  all  indicated  that 
a  key  goal  of  the  program  was  fostering  good  decision-makers  and  problem-solvers  who  can 
apply  what  they  learned  to  real-life  exercises. 

As  a  whole,  the  instructors  felt  that  the  online  course  fit  nicely  into  a  small  group 
instruction  model  and  strongly  complemented  the  Army’s  “crawl,  walk,  run”  philosophy  of 
learning.  In  effect,  Phase  la,  or  the  asynchronous  portion  of  the  course,  provided  the  basic 
foundation  (“crawl”),  Phase  lb,  the  synchronous  portion,  allowed  the  students  to  put  their 
knowledge  to  use  in  electronic  and  paper  formats  (“walk”),  and  Phase  II,  the  resident  portion, 
prompted  the  students  to  fully  apply  their  knowledge  and  skills  in  real-life  scenarios  (“run”). 
Interestingly,  two  instructors  were  retired  officers  who  were  highly  familiar  with  this  three-part 
training  philosophy.  Neither  claimed  much  difficulty  with  the  technology  nor  the  instructional 
methodology.  In  fact,  retired  officers  familiar  with  small  group  training  methods  and  the  overall 
philosophy  of  learning  espoused  here  may  be  the  best  suited  instructors  for  such  an  approach. 

Instructional  Techniques 

The  instructors  noted  that  certain  instructional  strategies  and  pedagogical  approaches 
were  useful  in  online  environments  to  facilitate  student  learning  and  to  encourage  participation. 
Whereas  the  asynchronous  phase  involved  more  directive  and  one-way  instructional  techniques 
aimed  at  learning  basic  concepts  and  information,  instructors  utilized  more  indirect  questioning, 
prompting,  and  nudging  in  the  synchronous  phase.  The  instructors  indicated  that  they  were 
genuinely  interested  in  student  progress  throughout  the  course  and  that  they  were  there  to  help 
students  succeed.  For  example,  they  sent  out  weekly  reminders  about  assignments,  used  indirect 
questioning  and  prompting  to  engage  and  involve  students,  and  attempted  to  place  each  person  in 
a  leadership  position  within  their  groups,  where  possible,  to  boost  their  confidence  levels. 

Instructional  tactics  such  as  selecting  students  to  be  in  charge  of  activities  were  intended 
to  boost  student  participation  during  the  synchronous  component.  Generally,  the  course 
instructors  felt  that  these  tactics  were  successful  in  achieving  that  goal.  The  instructors  also 
found  that  matching  weaker  students  with  strong  leaders  was  beneficial.  In  fact,  they  noted  that 
this  often  resulted  in  the  respective  groups  supporting  poor  performers  on  their  own  with  little 
help  requested  from  the  instructors. 

Perceived  Advantages 

The  course  instructors  praised  the  course  highly.  One  instructor  claimed  that  advantages 
of  the  Web-based  course  included  the  ability  to  provide  specific  and  detailed  feedback  on  student 
work,  greater  learning  and  application  of  knowledge  by  students,  and  more  standardization  of 
course  content.  The  other  two  instructors  stated  that  a  key  advantage  of  the  synchronous  course 
sections  involved  teaching  students  how  to  work  with  others  as  a  team  to  solve  a  problem.  In 
addition  to  problem  solving  and  teamwork,  they  contended  that  it  enhanced  students’ 
communication  skills.  Given  the  rise  of  communications  technology  and  the  need  for  team  skills 
in  most  work  settings,  claims  that  distance  technologies  have  a  positive  impact  on  such  skills  are 
important.  Other  advantages  mentioned  included  allowing  students  to  get  the  most  current  and 
updated  material,  providing  students  with  immediate  feedback,  and  equipping  reservists  with 
skills  and  training  equal  to  that  of  active  duty  soldiers.  As  expected,  they  also  mentioned  that  the 


16 


distance  course  provided  an  additional  avenue  for  those  who  wanted  to  advance  their  military 
careers. 

The  instructors  noted  that  students  benefited  from  both  asynchronous  and  synchronous 
technologies.  For  instance,  the  synchronous  component  offered  students  immediate  feedback, 
mentoring  throughout  the  process,  and  the  opportunity  to  interact  with  both  peers  and  the 
instructors.  Although  the  instructors’  responses  indicated  that  they  valued  the  “virtual  talk” 
aspects  of  the  VTOC  since  it  approximated  talking  face-to-face,  they  did  not,  however,  view  it  as 
beneficial  when  forced  to  use  it  as  a  means  of  instruction.  One  instructor  claimed  that  it  was 
important  to  let  students  know  what  positions  they  were  going  to  assume  in  the  VTOC 
beforehand  and  to  assign  them  specific  materials  to  study.  He  also  sensed  that  the  ability  to  call 
on  students  to  answer  questions  in  the  voice  chat  increased  student  participation.  This  instructor 
argued  that  most  learning  took  place  in  the  synchronous  and  resident  phases  rather  than  in  the 
asynchronous  phase. 

In  tenns  of  the  asynchronous  or  delayed  discussion,  all  instructors  claimed  that  students 
were  effectively  mentored  as  they  progressed  through  the  systems  and  learned  new  concepts  and 
ideas.  Moreover,  they  felt  that  feedback  was  promptly  received  during  this  phase  and  was  based 
on  progress  as  well  as  performance.  Most  of  this  feedback  was  provided  through  e-mail, 
however,  not  directly  in  an  electronic  portfolio  of  work  or  in  the  student  activity  records  within 
the  learning  management  system.  Despite  the  strong  reliance  on  e-mail,  the  instructors  found 
benefits  from  the  other  technologies  employed  in  the  asynchronous  phase  as  well.  Whereas  the 
synchronous  technology  was  useful  for  helping  students  learn  how  to  work  together  as  well  as 
how  to  apply  knowledge,  the  asynchronous  phase  was  a  more  directive  and  one-way  approach  to 
the  learning  of  basic  information.  In  effect,  the  asynchronous  phase  prepared  students  for  the 
synchronous  phase. 

Assessment  was  different  in  Phase  la  (asynchronous)  and  Phase  lb  (synchronous).  In  the 
asynchronous  phase,  objective  forms  of  measurement  (e.g.,  multiple  choice,  matching,  etc.)  were 
utilized.  In  the  synchronous  portion  of  the  course,  however,  grading  became  more  subjective  as 
instructors  evaluated  student  military  plans.  Consequently,  they  relied  on  criterion  scoring 
checklists  and  guides  to  evaluate  and  grade  the  student  products. 

Perceived  Disadvantages 

While  these  instructors  did  not  indicate  many  disadvantages,  they  did  report  fairly  high 
attrition  rates  compared  to  correspondence  courses.  They  sensed  that  part  of  the  problem  was 
that  they  lacked  mechanisms  to  control  the  size  of  content  modules  during  the  asynchronous 
portion  of  the  course.  As  a  result,  these  students  had  to  fit  a  fairly  robust  and  demanding 
curriculum  into  their  already  full  lives.  Early  modules  or  “volumes”  in  the  asynchronous  phase 
were  particularly  difficult,  according  to  the  instructors.  Those  who  made  it  to  the  second  volume 
usually  had  the  stamina  and  motivation  to  complete  the  entire  course. 

Overall  Impressions  and  Suggestions 

Overall,  the  instructors  enjoyed  teaching  the  course  online  and  using  the  technology. 

They  felt  that  not  only  were  the  students  going  through  the  distance  learning  course  better  trained 
than  those  taking  the  course  through  correspondence,  but  that  the  distance  learning  course 
provided  the  students  with  general  skills  such  as  problem-solving  and  group  communication  that 
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are  applicable  to  any  position  in  the  Army  and  could  not  be  gained  from  a  correspondence 
course.  In  fact,  they  recommended  eliminating  all  correspondence  courses  in  favor  of  those 
offered  via  distance  learning,  especially  for  Army  recruiters  and  commanders  spread  out  across 
the  United  States.  When  asked  about  advice  they  might  offer  regarding  similar  projects,  they 
suggested  focusing  on  tools  and  methods  that  foster  interaction,  providing  instant  or  at  least 
consistent  feedback,  and  utilizing  and  promoting  the  ability  to  post,  share,  and  reflect  on  student 
products. 


Interview  with  DL  Education  Advisor 

The  interview  with  the  DL  Education  Advisor  for  the  Armor  School  provided  some 
valuable  information  concerning  the  design  of  distance  learning  courses  based  on  her 
experiences  with  the  Armor  Captains  Career  Course  and  other  distance  learning  courses  within 
the  Army.  This  interview  also  confirmed  many  of  the  focus  group  findings  from  the  students 
and  instructors. 

Overall  Perspective 

Two  previous  meetings  with  the  DL  Education  Advisor  indicated  that  she  was  quite 
pleased  with  the  program  and  was  an  avid  supporter  of  it.  At  the  same  time,  she  was  interested 
in  additional  course  evaluation,  especially  as  it  might  improve  student  completion  rates,  help 
fine-tune  course  production  and  system  resources,  and  lead  to  enhanced  online  Web-based 
instruction  tools  and  strategies.  She  emphasized  the  fact  that  anyone  involved  in  the 
development  of  distance  learning  technology  must  be  flexible  and  adaptable  because  the 
technology  is  changing  so  rapidly  that  one  cannot  just  look  at  where  the  technology  is  now,  but 
must  also  consider  where  it  will  be  a  year  from  now.  Not  surprisingly,  the  course  tools  for  this 
program  were,  in  fact,  utilizing  many  current  hardware  and  software  technologies. 

Theoretical  Perspective 

The  DL  Education  Advisor  was  extremely  cognizant  of  learning  theory.  While  noting 
constructivist  (Duffy  &  Cunningham,  1996)  and  learner-centered  principles  (APA,  1993)  related 
to  flexibility,  variety,  choice,  meaningfulness,  performance  assessment,  and  learning  in  authentic 
contexts,  she  also  pointed  to  practical  exercises  that  they  attempted  to  embed  in  the  course  to 
help  students  learn  the  content.  She  emphasized  that  while  learning  key  terms  and  concepts  were 
important,  application  of  these  terms  and  concepts  was  the  ultimate  goal.  With  the  purpose  to 
train  these  students  to  command  companies  and  other  similar  duties  at  battalion  and  brigade 
levels,  it  was  imperative  to  focus  on  bottom-line  command  readiness.  In  effect,  student 
understanding  was  deemed  to  grow  from  use.  While  the  Internet  provided  the  mechanism  for 
course  delivery,  she  recognized  that  it  was  not  giving  them  everything.  Clearly,  the  learning 
environment  extended  beyond  the  Web. 

Caveats  and  Tips 

For  those  wanting  to  replicate  aspects  of  this  program,  the  DL  Education  Advisor 
provided  several  caveats  and  tips.  For  instance,  she  claimed  that  any  distance-learning  course 
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must  be  adaptable  and  flexible  to  changes  in  learner  needs,  content  requirements,  and  available 
technology.  While  one  must  remain  open  to  new  possibilities,  there  are  many  risks  involved  in 
exploring  and  selecting  a  particular  technology.  Consequently,  she  argued  that  leaders  must  be 
able  to  simultaneously  evaluate  current  technologies  for  student  learning  needs  as  well  as  those 
that  loom  on  the  horizon. 

To  help  others  in  comparable  roles,  and  for  those  who  are  designing  similar  programs, 
she  outlined  six  important  considerations  in  the  design  of  distance  learning  courses.  First,  she 
stated  that  all  courses  should  involve  direct,  e-mail  feedback.  Her  rationale  for  this  principle  was 
that  students  need  to  feel  connected  to  both  each  other  and  to  the  instructor  and  they  also  need  to 
have  a  way  to  assess  their  progress.  Second,  she  claimed  that  courses  should  have  meaningful 
content  that  allows  students  to  directly  apply  the  material  to  real-life  exercises.  Concerning  the 
AC3-DL  course,  meaningful  content  typically  involved  combat  situations  and  combat  readiness. 
Third,  there  should  be  minimal  extraneous  content  (e.g.,  extra  graphics  as  well  as  practice 
exercises)  so  as  to  limit  student  confusion  and  course  complexity.  Her  team  has  found  that 
students  will  skip  optional  or  periphery  materials  if  they  feel  cognitively  overwhelmed.  Fourth, 
designers  of  distance  learning  courses  should  carefully  analyze  their  target  audiences  so  they  can 
accurately  determine  what  they  want  and  need  out  of  a  course.  Fifth,  distance-learning  courses 
should  offer  flexibility,  choice,  variety,  meaningful  contexts,  and  performance  opportunities. 
Many  of  these  principles  relate  to  the  learner-centered  principles  from  the  American 
Psychological  Association  (APA,  1993).  According  to  the  DL  Education  Advisor,  it  is  vital  to 
create  an  active  learning  environment  with  a  balance  between  flexibility  and  learner 
accountability.  Not  surprisingly,  she  readily  admitted  that  the  instructor  was  a  key  part  of  that 
environment.  With  prompt  instructor  feedback,  students  were  not  isolated  in  their  online 
learning  endeavors.  Lastly,  designers  should  limit  their  visions  and  not  stretch  the  expectations 
of  technology  too  far  beyond  the  tools  and  options  that  have  been  proven  to  work.  The  DL 
Education  Advisor  argued  that  there  would  always  be  room  for  improvement  but  that  one  has  to 
start  somewhere. 

Perceived  Disadvantages  or  Problems 

The  DL  Education  Advisor  noted  several  problems  with  the  current  system.  First,  many 
students  wanted  printed  copies  of  the  course  materials.  Given  the  online  availability  of  the 
course  materials,  however,  she  felt  that  this  would  amount  to  an  excessive  waste  of  paper. 
Second,  the  learner-management  system  was  not  flexible  enough  for  most  of  the  students  and 
instructors.  For  instance,  as  alluded  to  earlier,  some  students  voiced  frustration  that  they  could 
not  move  on  to  another  volume  if  they  missed  too  many  items  on  the  gate  test.  Third,  since  there 
was  not  ubiquitous  access  to  the  Internet,  some  activities  and  events  may  not  always  have  been 
available  to  students  across  settings.  Fourth,  in  addition  to  Internet  access,  some  students  wanted 
access  to  course  materials  via  CD-ROM.  Access  to  materials  in  CD  format  is  problematic, 
however,  since  the  Army  would  lose  much  of  the  ability  to  track  student  progress  and  problems. 
The  DL  Education  Advisor  noted  that  the  Army  was  interested  in  knowing  that  students  truly 
learned  the  online  materials  and  were  obtaining  new  skills  and  competencies,  not  just  simply 
looking  up  answers  to  test  questions.  In  reflecting  on  overall  program  goals  and  expectations, 
she  admitted  that  this  was  primarily  an  issue  of  control. 
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Summary  of  Findings 


Web-based  Instruction  Advantages  and  Disadvantages 

As  detailed  in  Table  1,  across  the  students,  instructors,  and  the  DL  Education  Advisor, 
there  were  a  number  of  distinct  advantages  and  disadvantages  noted  during  the  focus  group 
sessions.  The  framework  of  Table  1  emerged  from  the  qualitative  data  collected  and  analyzed. 

Table  1.  Perceived  Advantages  and  Disadvantages  of  Web-based  Instruction 


Advantages 

Disadvantages 

Students 

1 .  Course  offers  flexibility  and 
convenience  to  those  working 
fulltime. 

2.  Students  can  work  at  own 
pace. 

3.  Immediate  feedback  and 
mentoring  can  be  received 
online  and  at  any  time. 

4.  Online  learning  environment 
can  be  structured  for  active 
learning. 

5.  Students  leam  to  work 
together  while  online. 

6.  Online  chats  fostered 
thoughtful  commenting  and 
reflection. 

1 .  Lack  of  a  pre-orientation  session  to 
detail  course  expectations  and  tools. 

2.  Length  of  course  components  did  not 
match  student  needs  and  abilities. 

3.  Novel  technologies  are  often  difficult  to 
use  and  tend  to  crash  or  impair  student 
computer  systems. 

4.  Lack  of  flexibility  in  the  system  forcing 
students  to  leam  in  a  preset  order.  In 
addition,  permission  of  instructor 
required  before  moving  to  additional 
modules  or  volumes. 

5.  Minimal  feedback  offered  in 
asynchronous  examinations  other  than 
scores  received. 

6.  High  attrition  fostered,  in  part,  by  large 
modules. 

7.  Sense  of  community  and  group  identity 
takes  significant  forethought  and 
planning  on  the  part  of  the  instructors. 

Instructor 

1.  Fits  existing  small  group 
model  of  instruction  used  by 
the  Army. 

2.  Could  tailor  instructional 
strategies  to  the  form  of 
content  delivery. 

3.  Capability  to  provide 
immediate  feedback  on 
student  work. 

4.  More  detailed  feedback. 

5.  Group  interaction  among 
students  could  help  boost  the 
confidence  of  lower 
performers. 

6.  Greater  learning  and 
application  of  knowledge. 

7.  More  standardization  of 

content. 

8.  Teach  students  how  to  work 
together  to  solve  problem 
situations. 

1 .  High  attrition  or  low  completion  rates. 

2.  Excessive  student  time  commitments. 

3.  Lack  of  instructor  control  over  size  of 
content  modules. 
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9.  Help  students  to  stay  up-to- 
date  with  their  knowledge. 

10.  Enhances  communication 
skills. 

1 1 .  Timely  opportunities  for 
online  mentoring. 

DL  Education 
Advisor 

1 .  Course  can  take  advantage  of 
recent  advances  in  technology 
and  theory. 

2.  Can  embed  different  forms  of 
media  to  address  different 
student  learning  style  needs. 

3.  Technology  tools  allow  for 
historical  tracking  of  student 
work  and  learning. 

4.  Authentic  and  meaningful 
activities  can  be  embedded  in 
online  learning. 

1 .  Many  risks  involved  in  selecting  a 
particular  technology. 

2.  Hard  to  be  aware  of  future  technologies 
while  dealing  in  present  technology 
problems  and  issues. 

3.  Too  many  choices  or  information 
sources  can  overwhelm  learners. 

4.  Some  students  wasted  paper  when  had 
electronic  versions  of  materials. 

5.  Some  students  asked  for  CD-ROM 
versions  of  the  course  materials  when 
already  had  Web  access  and  the 
potential  for  a  paper  version. 

6.  Learning  management  system  was  not  as 
flexible  as  needed. 

Web-based  Instruction  Considerations  and  Issues 

The  DL  Education  Advisor  focused  on  overarching  design  issues  or  guidelines.  In  a 
review  of  the  focus  group  and  interview  data  across  participants,  as  well  as  the  course  materials, 
reports,  and  handouts  provided  by  the  DL  Education  Advisor  and  her  staff,  there  were  ten  key 
design  considerations  or  issues  for  Web-based  instruction  consistently  mentioned  by  the 
students,  instructors,  and  supervisors.  These  ten  areas,  detailed  in  Table  2,  are  not  intended  to  be 
exhaustive  but,  instead,  are  simply  a  summary  of  key  findings  from  the  focus  group  discussions. 

Table  2.  Web-based  Instruction  Considerations  and  Issues 


Web-based  instruction 
consideration  or  issue 

Student  Advice 

Instructor  Advice 

DL  Education  Advisor 
Advice 

1.  Feedback. 

E-mail  is  important 
mechanism  for  contacting 
instructors. 

Provide  instant  and 
consistent  feedback  with  e- 
mail  and  other  tools. 

Involve  direct  e-mail 
feedback. 

2.  Meaningful 
and  Real- 
World 

Content 

The  construction  of  online 
products  should  approximate 
real-world  application. 

Require  students  to 
produce  products  that 
instructors  and  peers  can 
evaluate. 

Include  meaningful  content 
and  allow  students  to  apply 
new  skills  to  real-life 
exercises. 

3.  Size  and 

Scope  of 

Content 

Materials 

To  maintain  motivation  and 
increase  completion  rates, 
divide  asynchronous  content 
and  testing  into  smaller  units 
or  accomplishments. 

To  increase  student 
completion  rates, 
instructors  need  some 
control  to  change  the  size 
of  content  modules. 

Utilize  minimal  extraneous 
content,  graphics,  or 
practice  exercises. 
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4.  Course 

Development 

and 

Organization 

A  pre-orientation  session 
will  help  address  questions 
and  concerns  about  the 
online  course.  Students  need 
lecture  and  direct  instruction 
before  project  work. 

Learn  basic  content  in 
asynchronous  phase 
(“crawl”),  put  knowledge 
to  use  electronically  and  on 
paper  in  synchronous  phase 
(“walk”),  and  apply 
knowledge  in  real-life 
scenarios  in  residential 
phase  (“run”). 

Carefully  analyze  target 
audience  wants  and  needs 
prior  to  course 
development. 

5.  Role  of 
Instructor 

Instructor  is  helpful  as  a 
facilitator  of  learning.  The 
same  instructor  should 
support  students  across  all 
phases  of  online  training. 

Instructor  role  is  more  of  a 
facilitator  of  the  learning 
process;  providing  tools, 
means,  and  guidance  to 
learn  effectively.  Indirect 
questioning,  prompting, 
reminders,  role  playing, 
and  direct  requests  are 
ways  to  engage  and  involve 
students. 

Instructor  provides  feedback 
and  sense  that  someone 
cares  about  their  learning. 

6.  Small  Group 
Structuring 

In  online  role-play,  rotate 
roles  among  group  members. 

Match  stronger  leaders  and 
weaker  students  in  role- 
play  activities  to  boost 
performance  and 
confidence.  Provide 
instructions  and 
information  prior  to  online 
events  such  as  role  plays 
and  product  discussions. 

Create  active  environment 
with  role-plays  and 
simulations,  but  must 
provide  balance  between 
flexibility  and  learner 
accountability. 

7.  Flexible  and 
Active 
Learning 

Be  flexible  and  allow 
students  to  complete  online 
modules  at  their  own  pace; 
minimize  need  for  instructor 
to  certify  students  are  ready 
for  next  step  or  phase. 

Distance  learning  helps 

Army  Reserve  students  fit 
training  into  busy 
schedules  and  keep  up  with 
active  duty  personnel. 

Offer  flexibility,  choice, 
variety,  meaningful  contexts 
for  learning,  and  student 
performance  opportunities. 

8.  Technology 
Utilization 

To  minimize  frustration  and 
downtime,  utilize  basic 
functions  or  technologies, 
where  possible. 

Use  asynchronous 
communications  for 
learning  basic  concepts  and 
synchronous 
communications  for 
application. 

Limit  technological  visions 
and  begin  to  incorporate 
technology  based  on  what  it 
can  presently  accomplish. 

9.  Build 

General 

Skills 

Through 

Online 

Communicati 
on,  Problem- 
Solving, 
Teamwork, 
and  Identity 

Small  talk,  introductions, 
and  information  sharing 
helps  form  team  identity. 

Communication  skills, 
problem  solving,  and 
teamwork  are  general  skill 
outcomes  of  interactive 
distance  learning.  Online 
tasks  should  involve 
teaching  students  how  to 
work  with  each  other  on  a 
team  to  solve  a  problem. 
Teamwork  and  virtual  talk 
among  small  groups  fosters 
interaction  and 
participation. 

Courseware  structured  to 
move  from  individual  effort 
(asynchronous  component) 
to  application  exercises  in 
small  group  collaboration 
activities  (synchronous 
component)  to  problem 
solving  in  collective  efforts 
within  units  (resident 
component)  is  a  useful 
framework  for  fostering 
student  learning. 
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10.  Assessment 
Practices 


Online  assessments  should 
closely  match  real-world 
expectations.  Focus  might 
shift  from  quantity  of 
learning  or  breadth  across 
areas  to  quality  or  depth  of 
learning  in  particular  areas. 
Assessments  should  also 
cover  smaller  amounts  of 
instruction  or  learning. 


Asynchronous  learning  is 
more  ideally  suited  for 
objective  tests  and 
measurements,  while 
synchronous  might  be  used 
for  student  performances  or 
products  and  criterion 
referenced  evaluations. 


Online  assessments  can 
include  automated  pre-tests, 
post-tests,  and  practice 
exercises  that  provide 
immediate  student  feedback. 
Random  feedback  or 
assessment  tools  are  also 
beneficial.  While 
evaluation  gates  require 
application  of  learning,  the 
learning  management 
system  needs  to  be  more 
flexible  and  adaptable  in 
regards  to  items  missed  on 
gate  examinations. 


Many  of  the  above  considerations  and  issues  reflect  a  constructivist  and  learner-centered 
teaching  perspective  thought  to  be  important  in  the  use  of  collaborative  technologies  (Bonk  & 
Cunningham,  1998).  Similarly,  Bonk  and  Cummings  detail  12  guidelines  for  Web-based 
instruction  (e.g.,  providing  prompt  feedback,  giving  students  choice,  establishing  psychologically 
safe  learning  communities,  etc.)  that  directly  link  to  the  one  or  more  of  the  14  learner-centered 
principles  (American  Psychological  Association,  1993).  Furthermore,  in  a  review  of  Web-based 
support  systems,  Oliver  and  McLoughlin  (1999)  detail  a  number  of  ways  constructivist  practices 
can  be  defined  in  terms  of  Web-based  supported  tasks  and  processes  of  learning.  For  instance, 
the  goal  of  authentic  learning  might  be  realized  through  project-based  learning,  including  access 
to  databases  that  engage  students  in  real  world  tasks.  Along  these  same  lines,  the  goal  of  active 
learning  might  be  supported  through  the  creation  of  reports,  Web  pages,  or  student  online 
commenting  on  peer  work,  which  could  involve  processes  related  to  exploring  interests  and 
ideas,  comparing  and  synthesizing  resources,  and  revising  original  ideas.  In  effect,  Oliver  and 
McLoughlin  believe  that  the  combination  of  appropriate  instructional  activities  with  Web-based 
tools  and  resources  can  support  constructivist  types  of  learning.  However,  they  point  out  that 
many  questions  remain  related  to  how  to  develop  rich  Web  resources  such  as  the  AC3-DL  course 
while  supporting  student  active  learning  and  collaboration. 
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CONCLUSIONS 


There  were  many  distinct  advantages  as  well  as  disadvantages  related  to  this  distance 
learning  course.  The  results  from  this  research  indicate  that  online  learning  does  allow  for 
greater  and  timelier  feedback,  authenticity,  convenient  and  meaningful  learning,  and  increased 
communication.  On  the  other  hand,  too  many  choices  can  overwhelm  learners,  and  what  may 
appear  to  be  a  flexible  system  may  not  be  when  constrained  by  technology  downtime  or  lack  of 
system  familiarity.  Interestingly,  while  feedback  may  be  made  an  instructional  priority  and  a 
success  story  in  one  phase  of  online  instruction,  in  another  phase,  it  may  seem  lacking  to  online 
students.  Certainly  synchronous  events  appear  to  have  more  opportunity  for  a  sense  of  peer  and 
instructor  responsiveness  than  asynchronous  activities.  Instead  of  focusing  on  how  to  exploit  the 
benefits  of  synchronous  training,  however,  research  by  Hiltz  (1998)  suggests  that  the  success  of 
asynchronous  learning  may  hinge  on  the  degree  of  collaborative  learning  and  interaction  in  the 
course.  Regardless  of  the  distance  learning  technologies  employed  in  the  AC3-DL,  time 
commitments  were  a  prevalent  factor  impacting  student  participation  and  success  rates. 

As  indicated,  there  were  a  number  of  Web-based  instruction  principles  or  considerations 
that  emerged  during  the  interviews  and  focus  group  discussions.  Not  surprisingly,  these  areas 
related  to  the  role  of  the  instructor  as  the  facilitator  and  organizer  of  the  learning  process,  the 
need  to  embed  tasks  that  require  active  learning,  problem  solving,  and  teamwork,  the  caution  to 
not  simply  select  technologies  because  they  exist,  and  the  need  to  think  about  how  assessments 
may  vary  based  on  the  phase  of  learning  entered  into  as  well  as  the  technologies  available  to 
assist  and  assess  student  learning.  Additional  principles  or  topic  areas  addressed  the  need  for 
consistent  and  prompt  student  feedback,  meaningful  contexts  for  student  learning,  thoughtful 
structuring  of  group  or  team  activities,  and  extensive  course  planning  and  organization.  Whether 
these  are  the  primary  or  sole  ingredients  of  online  course  success  remains  to  be  seen. 

Clearly,  for  students  interested  in  moving  up  within  their  military  careers,  AC3-DL 
appears  to  be  a  successful  and  rewarding  online  course  experience.  It  was  interesting  to  discover 
how  novel  instructional  technologies  embedded  within  AC3-DL  activities  intersected  with  new 
forms  of  teaching  and  learning.  During  the  focus  group  sessions,  there  was  a  definite  feeling  of 
commitment  from  all  parties  involved.  What  caused  such  feelings?  First,  the  learners  had  a  goal 
to  complete  the  training.  Second,  the  instructors  were  highly  involved  and  enthused  about 
delivering  instruction  in  a  new  way.  Third,  the  DL  Education  Advisor  and  her  staff  were 
extremely  supportive  and  interested  in  student  success  within  the  program.  To  obtain  this 
success,  they  attempted  to  link  sound  instructional  design  principles  with  recent  inroads  in 
cognitive  psychology  as  well  as  advances  in  instructional  technology.  In  effect,  all  three 
parties — students,  instructors,  and  administrators — had  incentives  that  were  vital  to  student 
completion  and  program  success.  Despite  several  significant  obstacles  and  problems,  this  team 
was  successful  in  meeting  their  goals.  They  utilized  sound  instructional  design  ideas  and 
principles,  built  and  delivered  courseware  over  the  Internet,  allowed  for  geographically  dispersed 
students  and  instructors  to  collaborate  in  real  time  as  well  as  in  delayed  modes,  and  tracked, 
monitored,  regulated,  and  provided  feedback  on  student  progress. 

While  the  focus  group  discussions  and  interviews  revealed  several  problems  in  the  AC3- 
DL  course  that  slowed  or  hindered  student  completion  rates,  the  course  administrators  have 
already  implemented  a  number  of  changes  to  the  course  sequencing  that  have  enhanced  and 
accelerated  student  completion.  In  the  newer  version  of  the  course,  students  alternate  individual 
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work  within  the  asynchronous  courseware  with  a  weekend  of  collaborative  work  in  the  VTOC 
(i.e.,  synchronous  training).  With  this  format,  there  are  now  seven  weekend  VTOC  sessions 
instead  of  ten.  Many  of  the  focus  group  students  we  interviewed  were  actually  involved  in  the 
transition  to  the  new  AC3-DL  format.  According  to  the  DL  Education  Advisor,  if  students  keep 
up  with  the  content,  they  can  now  finish  the  program  in  12  to  16  months.  In  fact,  one  recent  “go- 
getter”  completed  it  in  just  9  months. 

The  AC3-DL  team  has  also  begun  supporting  students  with  expert  mentoring  in  the 
VTOC.  In  such  mentoring  sessions,  the  interface  is  slightly  different  so  that  guest  experts  and 
other  visitors  do  not  need  to  log  in  or  control  complex  collaborative  tools,  but  instead  meet  in  a 
3D  meeting  hall  that  simply  relies  on  text  chat  and  voice  conferencing.  While  visitors  can 
request  a  virtual  microphone,  the  team  currently  asks  most  visitors  to  pose  questions  and 
comments  via  the  text  chat,  thereby  leaving  the  voice  conferencing  for  the  guest  speaker.  Use  of 
the  VTOC  for  mentoring  has  the  potential  to  expand  the  instructional  capabilities  and 
responsiveness  of  the  program.  Consequently,  it  might  prove  valuable  to  explore  the  types  of 
mentoring  approaches  provided  as  well  as  student  preferences  and  attitudes.  In  addition, 
research  might  explore  how  questions  are  raised  and  addressed  as  well  as  how  the  content  of  the 
guest  mentoring  impacts  student  course  performance  and  motivation  to  complete  the  program. 

There  are  many  avenues  for  course  and  tool  development  as  well  as  student  testing  and 
evaluation  within  military  e-leaming  as  well  as  in  higher  education,  K-12,  and  corporate  settings. 
As  new  developments  unfold,  it  is  imperative  that  researchers,  scholars,  instructors, 
administrators,  and  politicians,  who  too  often  are  struggling  just  to  stay  abreast  of  developments 
in  their  own  field,  become  aware  of  common  findings  or  themes  within  e-leaming  research  and 
teaching  efforts  across  these  settings.  The  present  research  provided  one  look  at  the  advantages 
and  disadvantages  as  well  as  many  instructional  considerations  and  issues  within  a  unique  online 
learning  program.  Other  studies  might  explore  completion  rates,  attitudes,  and  overall  learning 
when  one’s  career  is  not  contingent  on  course  completion. 

Certainly,  the  military  setting  of  this  course  offered  a  unique  training  need,  extremely 
committed  instructional  designers  and  instructors,  highly  motivated  students,  timely  instruction 
and  feedback,  and  the  funding  to  build  and  employ  novel  technologies.  As  similar  online 
courses  are  generated  in  other  settings,  it  is  hoped  that  course  developers  and  online  instructors 
will  expand,  modify,  and  utilize  some  of  the  key  considerations  and  principles  mentioned  here, 
while  also  exploring  the  advantages  and  taking  significant  steps  to  limit  the  disadvantages. 
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